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ABSTRACT 

The advice of evaluators is useful ^ilv, the planning of 
proatam objectives because the program will eventually be evaluated 
A . an l.he basis of these obiectives- This paper providers two tools for 
the development of program objectives. The "Levels of Program 
Objectives" method focuises. on the intended outcomes of a program. The 
levels range frbn the resource, activity, and client learning/input 
vels to the instructional process and student outcome levels. The 
Objectives Checklist" method questions the clarity, significance, 
and the measurability of each program objective. These provide 
program objectives that are a meaningful basis for evaluation. 
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Program Objectives 
Stan Drezeky Dwain Estes, Alan Roecks, John Andrews 
Education Service Center » Region 20 ^ . 

Most federal and many state educational programs require program 

♦ 

staff to develop objectives which become the basis for program evalua- 

tion* Evaluators are often asked to assist in the program planning 

process* * This paper provides a framework for thinking about objectives 

and two concrete tools. These- tools » ,the "Levels of Program Objectives" 

and ^Objectives Checklist," are being used in the lauthorit^'^rk reviewing 

r 

program objectives for a large, intermediate educational service agency. 
They have been applied to over 80 educational programs and 500 objec-. 
tives^^ Several hundred of these objectives are now being implemented. 

« 

Running head: Program Objectives j 

I ■ . 

Author's note: This article is based upon a paper presented at the 
jl^ Southwest Educational Research Association annual meeting. 

San Ajitonio, February 7-8, 1980. 
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\ . Program Objectives 

Many educational programs require^^^^^jerctives as a basis for program 
evaluation despite the iDOvement in educational evaluation avay from 
objectives. The drift from ob'jf^ctives , however > was based on a very nar- 
row concept of objectives. By returning" to a broader meaning fo'r objec*- 
tives> focusing on priority objectives^ and using the tools provided 

m 

in this paper > the practicing evaluator>. can successfully assist in pro- 
gram planning and, if required to evaluate objectives > can have a meaning- 
ful basis for evaluation. # ^ , . 

Up until the mid^l960's three definitions of evaluation were common. 
Evaluation was defined as either: (1) nearly identical' to measurement > 
(2) the professional judgment of in^dependent expjert-p compar.ing the object^ 
evaluated against a set of criteria written by copimittees of experts^ - 
o^r O) the comparison of performance data, against prff^pecified objec- 
tives (Stufflebeam et al, 1971; Worthen & Sanders, 1973; Glass, Note 1). 
Between the early 1930'^ and the mid-1960*s, it was the'^objectives-based 
approach which dominated evaluation (Guba, Note 2). , • 

In the last decade new definitions and approaches to evaluation 
have emerged. As early as 1969, G'lass (Note 1) had evaluated the objectives- 
based apprpach as having reached the limits of its growth. More recently 
Webster and Stufflebeam (Note .5) reported for thirty-four largfe urban schodl 
districts that the new decision^oriented approach had the greatest effect 
on the development and operationalization of these districts' evaluation" 

activities. The Webster ^nd Stuff lebedjn paper labeled" objectives-abased 

V- 
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Studies as quasi-evaluation* 

. Proponents and opponents have debated the status *of objectives in 

program evaluation 'for years* Proponents assert that objectives help 

decide what content to teach^* design relevant instruction, and evaluate 

instruction (Bakeir> 197A; Popham, 1975). Opponents argue objectives 

require a Sbt.of time, emphasize formal and trivial as opposed to sub- 

stantive and important aspects of instruction, make us feel guilty about 

responding to circumstances not known at the time the objectives vere 

formulated, and alienate the student from intellectual activity (Popham, 

1975; Scriveri, 1972; Welsh & Hambleton, •1976)- 

What many reject in '^objectives''^ is hovj objectives have been opera- 

tionalized. Evaluation blossomed in response to the idea that educators 

tould "engineer" better education just the way scientists and technolo- 

« 

gists engineered the Space Program. Operating within the metaphor of 
scientific engineering the term "objectives" has come, to denote a focus 
on overly precise^ often unimportant, behavioral indices as opposed to 
general important , learning outcomes (Baker, 197A; Cooley, 1978; and 
McAshan, 1977). Ralph Tyler himself (Shane i Shane, 1973) commented on 
looking back on educational objectives over the years, that "objectives'* 
as a terrri had come to mean , something far more specific than its original , 
usage* 

x 

Wergin^s (1976) review of .the literature suggests the most important 
decision relating to a program is to invest organizational resources 
in the program and its objectives in the first place-. Instead/of overlv 
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precise and unimportant indices of program accomplishment, we must vrite 
prioritv objectives tha^t are the broad, important program outcomes. 
These are the outcomes which must be achieved or the board, advocacy 
group, special interest representation, or funding agency will surely 
question program funding. They are so vit^l that program staff believe 

w 

% 

•they must be accomplished. For a $30,000 program or program component, 
this might be just one objective. How do we develop these objectives? 
^ools to develop objectives 

Levels of Program Objectives, Figure 1, presents an idea which is 

basic to any systems theory positing inputs, processes, and outcomes 

and hierarchical systems. Many planning^ evaluation, ^and curriculum 
development **models" developed during the last twenty years have em- 
ployed these concepts. As simple as the levels idea is, it increases 

I the probability of meaningful objectives. One of -the most systematic 
implementations of 4this concept by an evaluation organizatiorh occurred 

. in the mid^l970s at Austin^ (Tx,) Independent School District (Note 3), 
Oui^p.aboration borrows heavily from that office's early work. 



Insert Figure 1 about here 




This simple approach focuses on students and the outcomes the program 
^tends students to evidence. We term this* level student out cogie . 
We divide/ the many hierarchical systems in education into two thread classes: 
support programs and instructional programs w^'^Bistructional programs are 
programs whose immediate purpose is to engage the student in learning. 
Support programs are all the other educational programs whose ftutcomes 

\ 
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facilitate instruction; however, they not^'directly teach students. 

Obviously the support programs themselves occur at many levels. For 

an example of hierarchical order among support progi:^ms considet the 

* . * * 

following: a university teacher traj^^ing program,- a school district 
personnel selection system, a Schodl district • inservice education syst^ 
and, finally, the instructional program. We belieVe educational plant 
niijg is fortunate in dealing with -just the two *hroad classes of instruc- 
tional and support programs at' this point.' 

we conceive the instructional program to include input , instruction- 
al process , and student outcome levels. We use these terms in the broad-^ 
est sense of familiar usage. Student outcome is student performance. 
The focus o^^n objective at this level is to define "the good which 
our students will evidence in their Uehavior, Instructional process 
means whatever occurs between the student and his instructors or instruc-- 
tional media. It includes all variety of instructional technology. The 
focus of an objective at thrp level is to define the unique, powerful 
instructional transaction that is to transfer the rich mix of inputs 
into important outcomes for students. Input means the^:money, staff, 
time,, and matetial resources dedicated to the instt^ction^i^^^^X^^cess to 
come. The focus of an objective at this level is in specifying the 
"key ingredient*' — the input that is so vital, e.g,, a minimum amount 
of student time, that its^absence* would preclude accom.plishing student^ 
outcomes, 

♦ 

Support programs %ave as their one purpose producing client learning 
and products which can become inputs to the instructional program . We 
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conceive these support programs to include resource ^ activity , and client 
learning/product levels.' These terms are synomnr^s for the^inputs, processes 
and outcomes of the instructional program. Again we are using these 
terms in their broadest, most familiar usage. CMent learning/product / 
means the knowledge, attitude, skills^ or tangible capital gained by the 



client or school system as a 'result of activities in the support program. 
The onl^ purpose of these gains are as inputs to the i^istructional program. 
Support program objectives at this level focus on the critical learning 

s V » > 

or products * For example > a training program might focus on a key instruc- 
tional'' competency , ^e.g.., level of classroom questioning. A resource^ 
provision program m.ight focus on providing a certain set of materials 
(product) to be used in thf instructional proces4v^ Acti vity means the 
training, assist/an<5[§ , information .'dissemina^tion^ facilitation, or manu*- 
facturing which the support program undertakes to transform its own 
resources into .learnings^ and products of genuine usefulness to the in- 



structional program. Resource means the money, people, time, and 
material resources of the support program. The best way to understand 
this terminology is to examine a set of priority objectives; Figure 2 
presents one set for a home-based^rogram for early childhood students. 
The instructional program's student outcome objective specifies increas- 
ed developmental rate as the behavior which students will evidence on 
the Learning Accomplishment Profile. Each "j^iority objective logically 
relates .to the accomplishment of this student outcome. 



^ ^- Insert Figure 2 about here 
• ■ ■ ■■ ' ' f— — - 
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The instructional process level specifies student completion of 

»• "*'♦ * ' 

activities in the Hoinebased Model as the priority objective. This pro- 
gram sees the activities comprising the Hoinebased Model al the roost 
powerful instructional transaction far increasing developmental rate. 
At the input level the key ingredient is a staff competency: trainee 
knowledge of the process for impletnenting Homebased Model activities • 
Without this knowledge the instructional process would fiil ,and develop- 
mental rate could ntit be increased. 

At th^^upport program's activity level the training of teacheris" 
to a certain criterion level'^on a well-defined posttest is the essential 
program activity. Without this training the teachers could not imple- 
ment Homebased Model activities. At the support program^ s yresotirce 
level the production of the training progr^., a tangible- product presum- 

ably worthy of dissemination > is the key program product. 

» 

Every program has in reality ihultiple objectives et all these< 
levels. What the -priority objectii^'e. concept does is. to force prograii 

« 

and evaluation staffs to focus on key objectives which, in a sense, 
is s>Tionymous with developing a theory about how the program operates. 
This theory is so basic to the -program that program staff would readily 
admit the implications of not accomplishing an objective. 

What have we found as we attempted to "apply these ideas? In general,' 
in applving the levels and priorttv idea to sets of objectives we found: 
(1) instrojctional and support programs' objectives* tended to enphasize 
process as opposed to outcome levels, (2) program staff tended to view 
all objectives of equal importance and, therefore, many priority objectives 
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were lost among tangential objectives , (3) pr3grain staff .tended to 

develop too many objectives, (A) resource and* input levels tended to get 

ignored, even when they were the most important tangible .residue o^ a 

program^ and' (3) the agency tended to repeat objectives iT^rti prograiD to 

« 

program so many programs vere reallv variations on the saiife model, and 

\ 

(6) as a consequence, objectives .were often "contehtless Furthermore , 

V « * » » 

we found at both the state and local agency level lack of support for ' 

clarifying objectives. We will never forget our first hard year trving 

to clarify and focus objectives and the cries '^Evaluation is 'trying 

# 

to run my prograai. " ' (R6ecks» Note 4). We use an Objectives Checklist 
to feed back to program staff our suggestions on their objectives. 
Tools to develop objectives: Obj ectlves Checklist 

Figure 3 is a copy of a completed Dbjectlj^s Checklist. We have 
found* five key que^tioijs need to be/asked about each prSs^rity objectiye 
.during its development. A draft objective addressed by vay of example 
is: **By May 31, 19'79, at least fifteen of the trainees will have ini- 
plemented the Homebased Model in their districts as evidenced by: 

^ . • ■' 

1) actions of the LLk substantiating this instructional arrange^ 
ment for at least ninety students, ' * * 

2) case records -substantiating completion of teaching activities, 

3) documented student growth 6ti a selected deve^opmehtal ir^strument. 



Insert Figure 3 about here 

^ The first question asks if the objective is clear. The best test 
o'f this question is to have two persons independently visualize the 
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intent and describe it to the reviewer..** In this, case ve found a coniuslon 
in intent. The objective was really intended to be implementation of 
training^ but it included a reference to student outcome* We ■ recommended 

dropping this aspects ' ^ • / 

•* > 

The second question asks if the objective^is significant. The test 
we use is asking ourselves if the objective relates to an identified need 
of our clients or a corresponding organizational goal* In this particu^ 
lar case we had two years of case notes ^documenting t^hree tQ five-year- 
•old handicapped' children in outlying districts whose parents would only 
have them^ served at home. We also had requests from six special edtica- 
tion directors for this ser\^ice. 

The third question uses ' tlm.. ABCD criteria (Note 6) to jud^e the • 

■ \ / ^ 

formal aspects of the objective. Usually considerable feedback is , 

needed on these- characteristics of objectives. Particularly troub.lesome 

is matching the behavior and degree 'to th^ level of the objective. 

Tne audience criteria often elicits client comments on cost^ef f ectiveness , 

Of course, the interpersonal relations between evaluation and program 
/ . 

staffs is the key to moving toward meaningful priority objectives. 

The practicing educational evaluator confronted with the task of 
assisting in fh^ development of objectives has the tools he needs. ^ 
Our experience substantiates the usefulness cf these tools in develop- 
ing priority program objeqtives. By returning t.o the meaning of objec- 
ts 

tives as defined in this paper evaluators will be helping to plan meaning 

i 

programs and if required, evaluating, weLf^stated , yorthwhile, practical 

t 

obj actives ♦ • « > 
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_ • ■ • • Program Objective's^ ? 

"Program clients will experience bcttey-planned , more successful pro-^i 
grams; That is-.the purpose of "evaluation . " < 
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4 e " Figure Captions 

Figure 1. Tj^'pplogy of objectives adapted from the Austin (Texa/) 

Independent School District Office of Research and Evaluation {lAoli^ey^ 
f 

Note 3). ' ^ ^ . 

* Figure 2. An example 5et of program objectives. 

. Figure 3. A completed Objectives Chacklist illustrating the 
objectives review process • 
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SUPPORT PROjGRAMS 
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INPUTS- 
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STUDENT OUTCOME LEVEL: 

% . 

By May 31, 1980 at least two-- thirds of ninety handicapped early childhood 
students will have a greater developmental- rate during the project year 
(given attendance of at least 10 hours per week) than their developmental 
rate before entering the program, as evidenced by a 10% increase in develpp- 
mental rate on the Learning Accomplishment Profile. 



INSTRUCTIONAL PROCESS- LEVEL: ^ 

By May 31, 1979 at least fifteen of the trainees will have impletoented the 
Homebased Model in their districts as evidenced by: (IX actions of the' LEIA 
substantiating this instructional arrangement for at least 90 students and 
(2) case records substantiating completion of student activities, 

-tlLi-ENT LEARKING/TNPU^f LEVEL:"* [' 

4 

Given at -least three follov-up consultant visits at least twenty trainees will 
know the pr-ocess for implementing Homebased Model activities as evidenced by 
damonstrating two--thirds of the behaviors on Checklist A (behaviors . derived 
' f roui pnodule posttest) as observed by project staffs 



ACTIVITY LEfe: > . ' ' 

By December 15^ 1979 *Homebased Model project staff will have t-rained at least 
30'EC5--H teachers using the. developed module as evidenced by (a) workshop 
, registers^ (b) 50th^ percentil^^ or greater ratings on the Workshop Evaluation 
Form, arrd (c) ^80% corre'ct items on the posttest. 

RESOURCE .LEVEL: 4. . ^ ... 

Bji^S-eptember 30, 1979 'Koraebased Model project s.taff will have developed a training 
;inodule for the Homebased- Pfrgram as evidenced, fc^ the inod\ile containing (a) specific 
* trainee objectives., (b) ^pre/posttests covering these objectives; (c) five days of 
^'activities leading to the acconiplishrnent of the objectives, and (d> lists of* 
supfplementary media. 
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PROGRA^I OBJECTIVES CHECKLIST 



PURPOSE: To Oocvr^ent Eviluitio/^ Services * su9pcst1ons on cUrlty^ ^IgnlftcA^ce , 4nC ftedsuretil 1 1ty. The 
«>e$rec stitnncnt U the rcsponstbt 1 Uy of Ev«lu«t1on Services. The reUtionship of objectives 
'to E5C-20*9O*U 4nd rt^lontl needs U the r^soonslblllty af PUnfiln9 ; Development. 
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